Interactive information processing method, device and medium

ABSTRACT

Disclosed are an interactive information processing method, an electronic device and a storage medium. The method includes establishing a position correspondence between a display text generated based on a multimedia data stream and the multimedia data stream; and presenting the display text and the multimedia data stream corresponding to the display text based on the position correspondence.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of International Patent Application No.PCT/CN2021/116826, filed on Sep. 7, 2021, which is based on and claimspriority to Chinese Patent Application No. 202011057348.6 filed with theChina National Intellectual Property Administration (CNIPA) on Sep. 29,2020, the disclosures of which are incorporated herein by reference intheir entireties.

TECHNICAL FIELD

Embodiments of the present disclosure relate to the field of computerdata processing technologies, for example, to an interactive informationprocessing method, a device and a medium.

BACKGROUND

At present, users may view corresponding audio-video frames based onmultimedia data streams, or view corresponding content based on textinformation converted by the multimedia data streams.

According to the above manner, the viewed multimedia data stream and theviewed text information are independent of each other, and thecorresponding multimedia data streams cannot be quickly positionedaccording to the text content, resulting in that the user needs tosearch for the video frame corresponding to the text content from themultimedia data streams, thus the search efficiency is low and the userexperience is poor.

SUMMARY

The embodiments of the present disclosure provide an interactiveinformation processing method and apparatus, a device and a medium toachieve a position linkage between a multimedia data stream and adisplay text, thereby improving the technical effect of the readingconvenience and high efficiency.

The embodiments of the present disclosure provide an interactiveinformation processing method. The method includes steps describedbelow.

A position correspondence is established between a display textgenerated based on a multimedia data stream and the multimedia datastream.

The display text and the multimedia data stream corresponding to thedisplay text are presented based on the position correspondence.

The embodiments of the present disclosure provide an interactiveinformation display method. The method includes steps described below.

A display text generated based on a multimedia data stream is displayedon a first display region of a target page.

The multimedia data stream is displayed on a second display region ofthe target page.

A position correspondence is provided between the display text and themultimedia data stream.

The embodiments of the present disclosure further provide an interactiveinformation processing apparatus. The apparatus includes acorrespondence establishment module and a content presentation module.

The correspondence establishment module is configured to establish aposition correspondence between a display text generated based on amultimedia data stream and the multimedia data stream.

The content presentation module is configured to present the displaytext and the multimedia data stream corresponding to the display textbased on the position correspondence.

The embodiments of the present disclosure further provide an electronicdevice, and the electronic device includes one or more processors and astorage apparatus.

The storage apparatus is configured to store one or more programs.

When the one or more programs are executed by the one or moreprocessors, the one or more processors implement the interactiveinformation processing method according to any one of the embodiments ofthe present disclosure.

The embodiments of the present disclosure further provide a storagemedium including a computer-executable instruction. When executed by acomputer processor, the computer-executable instruction is configured toexecute the interactive information processing method according to anyone of the embodiments of the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

Same or similar reference numerals in the drawings denote the same orsimilar elements. It should be understood that the drawings areschematic and that the components and elements are not necessarily drawnto scale.

FIG. 1 is a flowchart of an interactive information processing methodaccording to an embodiment of the present disclosure;

FIG. 2 is a flowchart of an interactive information processing methodaccording to another embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a target page according to anembodiment of the present disclosure;

FIG. 4 is a flowchart of an interactive information processing methodaccording to another embodiment of the present disclosure;

FIG. 5 is a structural diagram of displaying a target content displayedin correspondence with a mark on a timeline according to an embodimentof the present application;

FIG. 6 is a schematic diagram of highlighting a mark on a timeline afterthe target content is triggered according to an embodiment of thepresent application;

FIG. 7 is a flowchart of an interactive information processing methodaccording to another embodiment of the present disclosure;

FIG. 8 is a flowchart of an interactive information display methodaccording to an embodiment of the present disclosure;

FIG. 9 is a structural diagram of an interactive information processingapparatus according to an embodiment of the present disclosure; and

FIG. 10 is a structural diagram of an electronic device according to anembodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure are described in more detailhereinafter with reference to the drawings. Although some embodiments ofthe present disclosure are shown in the drawings, it is to be understoodthat the present disclosure may be implemented in various forms andshould not be construed as limited to the embodiments set forth herein;conversely, these embodiments are provided so that the presentdisclosure will be thoroughly and completely understood. It should beunderstood that the drawings and embodiments of the present disclosureare merely illustrative and are not intended to limit the scope of thepresent disclosure.

It should be understood that the various steps recited in the methodembodiments of the present disclosure may be performed in a differentorder, and/or in parallel. Additionally, the method embodiments mayinclude an additional step and/or omit performing an illustrated step.The scope of the present disclosure is not limited in this respect.

As used herein, the term “include” and variations thereof are intendedto be inclusive, that is, “including, but not limited to”. The term“based on” is “at least partially based on”. The term “one embodiment”means “at least one embodiment”; the term “another embodiment” means “atleast one another embodiment”; the term “some embodiments” means “atleast some embodiments”. Related definitions of other terms are given inthe description hereinafter.

It is to be noted that references to “first”, “second” and the like inthe present disclosure are merely intended to distinguish one fromanother apparatus, module, or unit and are not intended to limit theorder or interrelationship of the functions performed by the apparatus,module, or unit.

It is to be noted that references to modifications of “one” or “aplurality” mentioned in the present disclosure are intended to beillustrative and not limiting; those skilled in the art shouldunderstand that “one” or “a plurality” should be understood as “one ormore” unless clearly expressed in the context.

FIG. 1 is a flowchart of an interactive information processing methodaccording to an embodiment of the present disclosure. The embodiment ofthe present disclosure is suitable for establishing a position linkagebetween a multimedia data stream and a display text, so as to achievethe linkage between the display text and the multimedia data stream whenthe display text or the multimedia data stream is triggered to bedisplayed. The method may be executed by an interactive informationprocessing apparatus. The apparatus may be implemented by softwareand/or hardware, for example, by an electronic device. The electronicdevice may be a mobile terminal, a personal computer (PC) side, a serverterminal, or the like. The method provided in this embodiment may beexecuted by a client, by a server terminal, or by both.

As shown in FIG. 1, the method in this embodiment includes the stepsdescribed below.

In S110, a position correspondence is established between a display textgenerated based on a multimedia data stream and the multimedia datastream.

The multimedia data stream may be audio-video stream data correspondingto a real-time interactive interface, or video stream data in arecording screen video after a real-time interactive interface isrecorded. The real-time interactive interface is any interactiveinterface in a real-time interactive application scenario. The real-timeinteractive scenario may be implemented by means of the Internet and acomputer, for example, an interactive application program implementedthrough a native program or a web program, etc. An audio frame of themultimedia data stream may be processed, for example, audio informationis extracted from the audio frame to obtain text informationcorresponding to the multimedia data stream, and the text information isdisplayed on a target page. Correspondingly, the text informationdisplayed on the target page is the display text. In order to achievethe effect of a synchronous linkage between the display text and themultimedia data stream, the position correspondence between the displaytext and the multimedia data stream may be established. The positioncorrespondence may be determined based on the timestamp. For example, insome application scenarios, the display text may be obtained byperforming the speech recognition on speech in the multimedia datastream. In these application scenarios, a position of the speech, whichcorresponds to a certain paragraph of characters in the display text, inthe multimedia data stream may be used as a timestamp of the paragraphof characters, and a position correspondence between the paragraph ofcharacters and the corresponding speech may be established.

In S120, the display text and the multimedia data stream correspondingto the display text are presented based on the position correspondence.

For example, the display text and the multimedia data stream betweenwhich the position correspondence has been established may be displayedon the target page. Thus, when a triggering operation (e.g. a click,etc.) on a certain part of the display text is detected, a video framecorresponding to the display text may be displayed according to theposition correspondence.

In the technical scheme of the embodiments of the present disclosure,the position correspondence is established between the multimedia datastream and the display text, and when triggering the target content ofthe display text is detected, the multimedia data stream may be jumpedto the video frame corresponding to the target content, so that a usercan understand speech information of a speaking user by combining thevideo frame and the target content, thus improving the technical effectof the interaction efficiency.

FIG. 2 is a flowchart of an interactive information processing methodaccording to another embodiment of the present disclosure. On the basisof the preceding embodiment, the corresponding display text is generatedaccording to the multimedia data stream to establish a positioncorrespondence between the display text and the multimedia data stream.Technical terms identical to or corresponding to the precedingembodiments are not repeated here.

As shown in FIG. 2, the method includes the steps below.

In S210, an audio-video frame of the multimedia data stream is acquired,and a user identity of a speaking user corresponding to the audio-videoframe is determined.

The multimedia data stream refers to a data stream acquired from areal-time interactive interface. An interactive scenario achieved by aplurality of users based on the Internet or a computer is used as thereal-time interactive scenario, for example, a multi-participant videoconference, a live broadcast and the like. After the real-timeinteraction, to conveniently review the conference content or determinethe core idea corresponding to the speech made by one or severalspeaking users, the real-time interactive scenario may be recordedduring the real-time interaction process, for example, the videoconference content may be recorded. The recorded real-time interactivescenario is used as a recording screen video. Correspondingly, themultimedia data stream is acquired based on the recording screen video.The recording screen video includes not only audio information of eachspeaker, but also video information, that is, the multimedia data streamcorresponds to audio frames and video frames. The user identity is usedfor representing the identity of each user.

For example, a user corresponding to a terminal to which the audioinformation belongs may be determined based on the audio frames and thevideo frames of the multimedia data stream, and then the user identityof the speaking user corresponding to the audio frames and/or the videoframes may be determined.

In S220, a display text corresponding to the multimedia data stream isgenerated based on the user identity and the audio-video frame.

The literal expression corresponding to an audio frame may be determinedbased on the audio information, i.e., the audio frame, of the multimediadata stream. A video frame of the multimedia data stream may be adocument shared by an interactive user, a shared screen, etc.Information such as literal information and/or links on a displayinterface may be determined based on the video frame. The display textcorresponding to the recording screen video may be determined based onthe audio frame and the video frame. The correspondence may be acorrespondence between the timestamp of the multimedia data stream andthe timestamp in the display text. The target page may be a pagedisplayed on the client. The page may include the multimedia data streamand the display text.

For example, the audio frames and the video frames of the multimediadata stream may be processed to obtain the literal expressioncorresponding to each audio frame and/or each video frame. The displaytext corresponding to the multimedia data stream is generated based onthe user identity and the timestamp corresponding to each audio-videoframe. To achieve a linkage between the display text and the multimediadata stream, the position correspondence between the multimedia datastream and the display text may be established. For example, themultimedia data stream is acquired based on the recording screen video.When it is detected that a progress bar of the recording screen video onthe target page is triggered, a playback moment corresponding to thecurrent video frame may be determined, and a text content correspondingto the playback moment may be displayed differentially in the displayedtext.

In S230, the audio-video frame of the multimedia data stream isacquired, and the user identity of the speaking user corresponding tothe audio-video frame is determined.

In S240, the display text and the multimedia data stream correspondingto the display text are presented based on the position correspondence.

The technical scheme of the embodiments of the present disclosure, byprocessing the audio-video frame of the multimedia data stream, the textinformation corresponding to the multimedia data stream may be generatedand displayed on the target page. Through the position correspondencebetween the multimedia data stream and the text information displayed onthe target page, a timestamp synchronization correspondence between themultimedia data stream and the display text is achieved, and thesynchronization between the text reading and video watching of users isimproved, thus improving the user experience.

In this embodiment, determining the user identity of the speaking usercorresponding to the audio frame based on the audio frame of themultimedia data stream includes determining the user identity of thespeaking user by performing the voiceprint recognition on the audioframe; and/or, determining a client identity of a client to which theaudio frame belongs, and determining the user identity of the speakinguser based on the client identity.

Generally, the voiceprint recognition is a kind of biometricrecognition, which is used for identifying the identity according toacoustic characteristics of the speaking user. Since the language ofeach speaking user has a unique voiceprint, speaking users withdifferent identities may be identified accordingly. Of course, theidentity of the client to which each speaking user belongs may also becollected, and the user identity of each speaking user is determinedbased on the identity of the client.

For example, the acoustic characteristic processing may be performed onthe audio frames in the multimedia data stream, and the user identity ofeach speaking user is determined according to a processing result.Generally, a client has its corresponding client account or clientidentity (ID) for distinguishing different clients. Therefore, theclient to which each speaking user belongs is acquired, and the useridentity corresponding to each speaking user may be determined based onthe identity of the client. However, when multiple users speak on acertain client at the same time, the multiple users cannot bedistinguished through the client ID. Therefore, the voiceprintrecognition may be further performed on voice data of each user, and thevoice of each person has a unique voiceprint, so that different userscan be distinguished according to the voiceprint. Then the multiple usesmay be marked as a client ID-user A and a client ID-user B,respectively, so as to distinguish identity information of differentspeaking users on the same client, that is, the user identity.

In this embodiment, the identity information of different speaking usersis determined. After the audio information of each speaking user isconverted into characters, the speaking user corresponding to theliteral content may be determined, thereby facilitating other users todetermine the core idea expressed by the speaking user corresponding tothe speaking content, thereby achieving the technical effect ofimproving the interaction efficiency.

On the basis of the preceding technical schemes, generating the displaytext based on the user identity and the audio-video frame includesobtaining a literal expression corresponding to the audio frame byperforming the speech-to-text processing on the audio frame, andgenerating a first display text in the display text based on the literalexpression and the user identity; and obtaining characters in the videoframe by performing an image-text recognition on the video frame, andgenerating a second display text in the display text based on thecharacters and the user identity.

For example, the audio frame in the recording screen video may beconverted into the corresponding literal expression based on aspeech-to-text module. The first display text in the display text may begenerated based on the identity of the speaking user corresponding tothe literal presentation and the audio. That is, the display textincludes at least one display text, and each of the at least one displaytext may display the corresponding content. For example, the contentdisplayed in the first display text may be characters generated based onthe audio frame. After characters in a video image are determined basedon the video frame, the second display text in the display text may bedetermined based on the characters in the video frame, that is, thesecond display text displays the content corresponding to the videoimage. Exemplarily, if the speaking user A shares a screen, the literalcontent in the screen may be recognized, and the literal content isdisplayed in the second display text.

It should be noted that the processed content for the audio frame andthe video frame may be uniformly displayed in the same text, so thefirst display text and the second display text are the same.

In this embodiment, the content corresponding to the audio-video frameis displayed in the display text, and the audio frame of each speakinguser in the recording screen video may be intuitively displayed ascorresponding characters, thus improving the convenience of determiningthe speaking content of the speaking user.

On the basis of the preceding technical schemes, obtaining the literalexpression corresponding to the audio frame by performing thespeech-to-text processing on the audio frame, and generating the firstdisplay text in the display text based on the literal expression and theuser identity include determining the literal expression correspondingto the audio frame, a timestamp currently corresponding to the audioframe and a user identity of a speaking user to which the audio framebelongs, and generating one piece of display content in the display textbased on the user identity, the timestamp and the literal expression.

It should be noted that the display text includes at least one piece ofdisplay content. When the speaking user speaks, speech information ofthe speaking user may be converted into a corresponding literalexpression. One piece of display content in the first display text maybe generated based on a timestamp of the speech information of thespeaking user and literal information corresponding to the speechinformation.

For example, for continuous audio frames, when the continuous audioframes belong to the same speaking user, the literal expressioncorresponding to the continuous audio frames, a user identitycorresponding to the continuous audio frames and timestampscorresponding to the continuous audio frames are determined, and onepiece of display content in the first display text may be generated.

It should be noted that the first display text may include multiplepieces of display content, each piece of the display content includes aspeaking user identity, a speaking timestamp and specific speakingcontent, and the user identity corresponding to each piece of displaycontent may be the same or different, but the timestamp corresponding toeach piece of display content is different.

In order to improve the readability of the display content in thedisplay text, one piece of display content may include at least twodisplay paragraphs. For example, obtaining the literal expressioncorresponding to the audio frame by performing the speech-to-textprocessing on the audio frame, and generating the first display text inthe display text based on the literal expression and the user identityinclude generating a next paragraph in the display content based on thelatter audio frame in the process of performing the speech-to-textprocessing on the audio frame, when it is detected that an intervalduration between adjacent audio frames is greater than or equal to apreset interval duration threshold and a user identity of a latter audioframe of the adjacent audio frames is not changed.

That is, when the speech-to-text processing is performed on the audioframe, the interval duration between two adjacent audio frames may beacquired. When the interval duration is less than the preset intervalduration threshold, it represents that two audio frames are continuousaudio frames. When the interval duration is greater than or equal to thepreset interval duration threshold and the user identity of the latteraudio frame of the two adjacent audio frames is not changed, itrepresents that the speaking user is not changed and only a pause existsin the speaking process. Based on the content corresponding to thelatter audio frame, it may be determined whether the contentcorresponding to the latter audio frame is generated into the nextparagraph in the display content. That is, after the speech contentcorresponding to the later audio frame is detected, the speech contentmay be combined with the speech content corresponding to previous audioframes for performing the semantic analysis, and the speech content maybe used as one paragraph or two paragraphs according to an analysisresult. This can facilitate other users to understand the speech contentof the speaking user, thereby improving the high efficiency ofunderstanding the speech content.

In order to quickly determine the speaking user corresponding to thedisplay content in the first display text, not only an identity of auser may be determined, but also a user avatar corresponding to eachspeaking user may be determined. Based on the user avatar, the speakinguser corresponding to the literal content in the display text may befurther quickly determined. For example, a user portrait correspondingto the user identity is acquired, and one piece of display content inthe display text is generated based on the user portrait, the useridentity, the timestamp and the literal expression.

It can be understood that each piece of display content may include theuser avatar, the speaking timestamp (corresponding to a timestamp of thespeaking user speaking in the recording screen video), and characterscorresponding to the audio information.

Of course, if the multimedia data stream is the corresponding videoframe of screen sharing, the content in the shared screen may beacquired, for example, the image-text recognition may be performed onthe video frame to obtain the second display text in the display text.For example, if the video frame includes at least one uniform resourcelocator (URL) address, the first display content in the second displaytext is generated based on the at least one URL address. If the videoframe includes a character, the second display content in the seconddisplay text is determined based on the character.

That is, after the multimedia data stream is acquired, the audio frameof the media data stream may be processed based on the speech-to-textmodule to obtain the literal expression corresponding to the audioinformation. To conveniently acquire information such as the screencontent, document and link shared by the speaking user in the multimediadata stream, the image-text recognition may be performed on each videoframe in the recording screen video to acquire the character and thelink in the video frame. The display content in the second display textin the display text is based on the character, the link, the identity ofthe speaking user and a timestamp corresponding to the content.

In the process of a practical application, there is a case that usersfilter the target content according to keywords, therefore, in theprocess of generating the display text based on audio-video information,the method further includes determining a third display text in thedisplay text based on the audio-video frame to determine a contentcorresponding to a target content from the first display text inresponse to detecting that the target content in the third display textis triggered, and displaying the content differentially. The thirddisplay text includes at least one keyword and/or at least one keysentence.

That is, in the process of converting the audio-video frame into thecorresponding characters, a keyword and/or a key sentence may beextracted from the converted character content, and the keyword and/orthe key sentence may be used as the third display text in the displaytext. In this embodiment, extracting the keyword and/or the key sentencemay be that at least one vocabulary is obtained by performing thetokenization processing on the content in the display text and a wordvector corresponding to each of the at least one vocabulary may bedetermined. According to the word vector corresponding to each of the atleast one vocabulary, an average word vector is determined, a similarityvalue between the word vector of each vocabulary and the average wordvector is calculated, and a vocabulary whose similarity value is higherthan a preset value is used as the keyword. The key sentence in the textmay be extracted in the same manner. A relationship between the firstdisplay text and each keyword and/or each key sentence in the thirddisplay text is established. When the key word or the key sentence istriggered, a position of the keyword or the key sentence in the firstdisplay text may be quickly determined from the first display text.Since the position correspondence is established between the displaytext and the multimedia data stream, when a sentence to which thekeyword belongs in the first display text is triggered, the recordingscreen video may play the multimedia data stream corresponding to thetimestamp, based on a timestamp of the sentence to which the keywordbelongs.

In the process of the practical application, to achieve a reasonablelayout, the quick determination of the display content corresponding toeach region in the target page may be the determination of a regionproportion of each display text on a display page, based on the displaycontent in each display text.

For example, the region proportion of each display text on the displaypage may be dynamically determined according to the display contentamount of each display text. For example, the more the display contentof a region is, the larger the region on the target page is.

The first display text, the second display text, the third display textin the display text and the multimedia data stream are respectivelydisplayed in preset display regions on the target page. A preset displayregion of the first display text is larger than a preset display regionof the recording screen video, the preset display region of therecording screen video is larger than a preset display region of thesecond display text, and the display region of the second display textis larger than a display region of the third display text.

That is, the first display region is mainly used for displaying textdata corresponding to the multimedia data stream, so that there is morecontent, and more regions may be acquired from the target page todisplay the first display text. An area of a second display region issmaller than an area of the first display region and is used fordisplaying the multimedia data stream such as the recording screenvideo; a display area of the third display region is smaller than thedisplay area of the second display region and is used for displaying thecontent corresponding to the video frame in the multimedia data stream;and a fourth display region is a region for displaying the third displaytext and is used for displaying the keyword and/or the key sentenceextracted from the multimedia data stream. Since the multimedia datastream generally corresponds to a relatively large number of characters,the large number of characters may be displayed in a relatively largefont on the target page in order to be easy for users to read; and thecontent of key sentences or keywords is less, the display region for thecontent on the display page may be as small as possible.

To clearly understand the content displayed on the target page, as shownin FIG. 3, the target page may include a control 1, a control 2, acontrol 3 and a control 4. The control 1 is a control for displaying thefirst display text, and the speaking content corresponding to eachspeaking user may be displayed in a text region of the control 1. Thecontrol 2 is a control for displaying the recording screen video, andthe recording screen video may be placed in the control 2. The control 3is a control for displaying the second display text, and informationsuch as a shared document and a shared link may be displayed in thecontrol 3. The control 4 is a control for displaying the third displaytext, and the keywords or key sentences corresponding to the recordingscreen video and the first display text may be displayed in the control4.

FIG. 4 is a flowchart of an interactive information processing methodaccording to another embodiment of the present disclosure. On the basisof the foregoing embodiments, there is a case that a language type usedby each speaking user speaking in the multimedia data stream is notmatched with a current user. Exemplarily, the language type used by eachspeaking user in the recording screen video includes English andChinese. If the language type used by the current user is Japanese, thecontent in the display text and the recording screen video cannot beread or understood. To reduce this kind of situation, a target languagetype corresponding to a target user (the current user) may be determinedin advance. Terminological explanations identical to or corresponding tothe above embodiments will not be repeated here.

As shown in FIG. 4, the method includes steps described below.

In S410, a user identity of a speaking user corresponding to theaudio-video frame is determined based on the audio-video frame of themultimedia data stream.

In S420, a target language type is determined, and the audio-video frameis converted into a literal expression corresponding to the targetlanguage type.

The audio-video frame in the multimedia data stream may be convertedinto a corresponding language type, and the converted language type isused as the target language type. For example, if it is required toconvert an audio frame of the recording screen video frame into thelanguage type of Japanese, the target language type may be Japanese.

In this embodiment, determining the target language type may be asfollows: acquiring a historical language type used by the currentclient, and determining the target language type based on the historicallanguage type.

For example, the historical language type selected by the currentclient, such as a language type used by the current client before thistime, is acquired. The target language type is determined based on thehistorical language type.

In this embodiment, determining the target language type based on thehistorical language type includes at least one of a use frequency ofeach historical language type, a language type used in the case ofconverting a previous audio frame into characters, a language type setby the current client, or a login address of the current client, thatis, determining the target language type based on the login address.

For example, the use frequency of each historical language type by thecurrent client is acquired, and a language type with the highestfrequency may be used as the target language type. For example, thehistorical language types used by the current client include Chinesewith 5 times, English with 2 times and Japanese with 3 times, andChinese may be used as the target language type based on the usefrequency of each historical language type. Alternatively, the languagetype used in the case of converting the previous audio frame into thecharacters may be acquired, and the language type previously used isused as the target language type. Alternatively, it may be determinedwhether a default language type is set in the current client, if thedefault language type is set in the current client, the default languagetype is used as the target language type. Alternatively, the loginaddress of the current client is acquired, a region to which the loginaddress belongs is determined, such as China, then Chinese may be usedas the target language type; and if the region to which the loginaddress belongs is the United Kingdom, English may be used as the targetlanguage type.

It should be noted that if the user does not trigger a language typeselection control, an original language type of each speaking user maybe acquired. The audio information is directly translated intotranslation data in the original language type, and the user identityand the translation data are used as the display content in the displaytext. The original language type of the speaking user here may be, forexample, the language type used by the speaking user when speaking.

On the basis of the technical schemes, to facilitate a user to adjustthe language type corresponding to the user in time, the displayinterface further includes a language type setting control. For example,referring to FIG. 3, in response to detecting that a language typeselection control 5 is triggered, a language selection list may bepopped up and includes at least one language type, and from which theuser may select a language type with a relatively high matching degreeas the target language type. After the target language type is selected,the display text on the target page may correspond to the targetlanguage type.

In S430, the display text is generated based on the literal expressionand the user identity, the position correspondence between the displaytext and the multimedia data stream is established, and the display textand the multimedia data stream are displayed on the target page.

For example, the display text corresponding to the recording screenvideo is generated based on the literal expression, the user identityand the character expression corresponding to each audio-video frame.

According to the technical schemes of the embodiments of the disclosure,the display text corresponding to the recording screen video can bedisplayed as the character expression corresponding to the targetlanguage type by acquiring a preset target language type, which not onlyimproves the convenience for users to read the display text, but alsocan locate a corresponding position in the recording screen video basedon the display text. Further, in a process of reading the display text,the target language type may also be updated based on the language typeselection control on the page, further improving the matching degreewith the users and the user experience.

On the basis of the technical schemes, the method further includesacquiring a search content edited in a search content editing control,and acquiring a target content corresponding to the search content fromthe display text, each target content is the same as the search content;and displaying the target content differentially in the display text,and marking an audio-video frame corresponding to the target content ina controlling control corresponding to the multimedia data stream. Thecontrolling control is displayed associated with the playback timelineof the multimedia data stream. A position of the audio-video framecorresponding to the target content is marked in the controlling controlbased on a timestamp of the target content.

The search content editing control may be a control displayed on thetarget page and configured for editing the search content. The targetpage may include display texts generated based on speech information ofdifferent language types. A server may acquire the search content editedin the search content editing control and use the acquired searchcontent as the target content. For example, the search content edited inthe search content editing control is “algorithm”, and the targetcontent acquired by the server is algorithm. Differentially displayingmay be displaying the target content in bold to distinguish the targetcontent from other content. Marking in the controlling control may bemarking the audio-video frame corresponding to the target content. Forexample, after the audio-video frame corresponding to the target contentis determined, a position of the audio-video frame in the controllingcontrol may be determined, and a dot or a triangle is marked below theposition, referring to FIG. 5. The audio-video frame corresponding tothe target content in the multimedia data stream is determined based ona pre-established association, and the audio-video frame correspondingto the target content is marked on the controlling control. In thismanner, the user can clearly know the position of the target content inthe display text and the position of the target content in themultimedia data stream.

It should be noted that in the search content editing control, thenumber of target content may be displayed, for example, the total numberof target content displayed in the search content editing control is 12,referring to FIG. 5.

It should also be noted that the number of target content may be morethan one, and correspondingly, the number of marks on the timeline maybe more than one. Referring to FIG. 3, the number of target content is12, and the number of marks on the timeline is also 12. Of course, tofacilitate the user to determine which piece of content in all targetcontent is currently triggered, the search content editing control alsodisplays the order corresponding to the currently triggered targetcontent.

In this embodiment, marking the audio-video frame corresponding to thetarget content in the controlling control corresponding to themultimedia data stream includes: determining a playback timestampcorresponding to the target content, and marking a video playback momentcorresponding to the target content in the controlling controlcorresponding to the multimedia data stream according to the playbacktimestamp.

For example, the playback timestamp corresponding to the target contentmay be determined, a corresponding multimedia data stream may bedetermined according to the playback timestamp, and a positioncorresponding to the playback timestamp is marked in the controllingcontrol.

In this embodiment, the method further includes, in response todetecting a triggering operation triggering the target content,determining a target playback timestamp of the target content; anddisplaying a marker corresponding to the target playback timestampdifferentially.

For example, the user may trigger the target content, when the targetcontent is triggered, the target playback timestamp corresponding to thetarget content is determined, and the marker corresponding to the targetplayback timestamp is displayed differentially from other markers.Differentially displaying may be color-differential displaying,highlight displaying, etc.

Exemplarily, referring to FIG. 6, when the user triggers the targetcontent corresponding to a marker 1, a target playback timestampcorresponding to the target content of the marker 1 may be determined.According to the target playback timestamp, it may be determined that amarker corresponding to the target playback timestamp on the timeline isthe marker 2, and this marker may be highlighted.

FIG. 7 is a flowchart of an interactive information processing methodaccording to another embodiment of the present disclosure. On the basisof the foregoing embodiments, after the content displayed on the clientdisplay interface is determined, when a control on the target page istriggered, a corresponding operation may be performed. Technical termsidentical to or corresponding to the preceding embodiments are notrepeated here.

As shown in FIG. 7, the method includes steps described below.

In S510, the user identity of the speaking user corresponding to theaudio-video frame is determined based on the audio-video frame of themultimedia data stream.

In S520, the display text is generated based on the user identity andthe audio-video frame.

In S530, the position correspondence is established between the displaytext and the multimedia data stream.

In S540, the display text and the multimedia data stream correspondingto the display text are presented based on the position correspondence.

In S550, in response to detecting a triggering operation triggering adisplay content in the display text on a target page, the multimediadata stream is adjusted to jump to a video frame corresponding to atimestamp based on the timestamp corresponding to the display content;and/or, in response to detecting a triggering operation for themultimedia data stream, a playback timestamp of the multimedia datastream corresponding to the triggering operation is acquired, and thedisplay text is jumped to the display content corresponding to theplayback timestamp in the display text based on the playback timestamp.

It should be noted that since there is a certain position linkagerelationship between the multimedia data stream and the display textdisplayed on the target page, a synchronous display between themultimedia data stream and the display text on the target page can beachieved.

Exemplarily, referring to FIG. 4, in response to detecting a triggeringoperation on the display content corresponding to a speaking user Andyin the first display text on the target page, based on the timestamp of00:00:05 corresponding to the display content, a progress bar of themultimedia data stream may jump to a playback moment corresponding tothe timestamp of 00:00:05. That is, when a user triggers a sub-control 6in the control 1, the progress bar of the multimedia data stream mayjump to the playback time of the timestamp corresponding to thesub-control 6. Correspondingly, in response to detecting a triggeringoperation triggering the progress bar on the multimedia data stream, theplayback moment of the multimedia data stream may be determined, andbased on a current playback moment of the multimedia data stream, thedisplay text may be jumped to the literal expression corresponding tothe current playback moment in the display text to achieve a linkagebetween the characters and the multimedia data stream.

On the basis of the preceding schemes, it should be noted that thedisplay content corresponding to the multimedia data stream is displayeddifferentially in the display text based on the position correspondence.

For example, the multimedia data stream may be a recording screen video.In the process of playing the recording screen video, the text contentcorresponding to a current video frame is displayed differentially inthe display text, so that the user can know the specific position of thecurrent video frame in the display text, thus facilitating the user toread.

In this embodiment, differentially displaying may be presenting the textcontent corresponding to the current video frame in a manner differentfrom other content in the text. Presenting the text contentcorresponding to the current video frame may be, but not limited to, thepresentation region and/or the presentation style. The presentationregion may be a presentation region in the target page that may attractthe attention of the user. The presentation style may be any at leastone of highlight displaying, bold displaying of the text contentcorresponding to the current video frame, and displaying a font colordifferentially from the font color of other content.

To achieve the precise adjustment of the multimedia data stream, thetarget page may also include a controlling control. Referring to FIG. 3,a controlling control 7 may also be configured to adjust a currentplayback moment of the multimedia data stream. To achieve the preciseadjustment, the length of a progress bar corresponding to thecontrolling control 7 is larger than the length of a progress bar in themultimedia data stream. Since a playback duration corresponding to themultimedia data stream is fixed, the longer the length of the progressbar is, the smaller the corresponding change amount is, that is, thehigher the corresponding adjustment precision is. Therefore, the lengthof the progress bar corresponding to the controlling control 7 is largerthan an entry bar corresponding to the multimedia data stream, thus theadjustment precision corresponding to the controlling control 7 ishigher than the adjustment precision of an adjustment control for theprogress of the multimedia data stream in the multimedia data stream.

For example, in response to detecting a triggering operation triggeringthe controlling control on the target page, a playback timestampcorresponding to the controlling control is acquired, the multimediadata stream is adjusted to jump to a playback position corresponding tothe playback timestamp based on the playback timestamp, and the contentof a display text of a video frame corresponding to the playbacktimestamp is displayed differentially in the display text.

For example, in response to detecting the triggering operationtriggering the controlling control on the target page, the playbackmoment corresponding to the progress bar in the controlling control maybe acquired, that is, a position to which the progress bar in thecontrolling control moves may be dragged, and the position maycorrespond to the playback timestamp of the multimedia data stream.Based on the playback timestamp, the multimedia data stream is jumped tothe audio-video frame corresponding to the playback timestamp, and thedisplay content corresponding to the video frame in the display text isdetermined at the same time.

It should be noted that in the playback process of the multimedia datastream, to facilitate the user to determine the text contentcorresponding to the current video frame, that is, the text contentcorresponding to the current video frame is quickly determined from thedisplay text. For example, in the playback process of the multimediadata stream, based on a playback timestamp corresponding to the currentaudio-video frame, the display content corresponding to the playbacktimestamp in the display text is determined, and the display content isdisplayed differentially, such as highlight displaying. In this way, thedisplay content corresponding to the current video frame may be quicklylocated from the display text, thus improving the convenience andefficiency of the user reading.

For example, in response to detecting a triggering operation triggeringan editing control on the target page, a permission editing list isdisplayed, and a user permission of each interactive user is determinedbased on the permission editing list. The user permission includes aread permission and an editing permission, the read permission is toread text information and video information, and the editing permissionis to edit information in a text.

It should be noted that the target page further includes an editingcontrol. If a user triggers a permission editing control, a drop-downmenu corresponding to the editing permission may be displayed, and theuser may edit the user permission of each speaking user in the drop-downmenu. The user permission may include the read permission and theediting permission. The read permission is to only browse or read avideo or a display text on the target page, but cannot edit the contenton the target page. A user with the editing permission can not only editthe content of the display text on the target page, but also share thecontent with other users. In this way, corresponding permissions can beset for different users, thus effectively avoiding the outflow ofinternal resources.

Exemplarily, referring to FIG. 3, in response to detecting a triggeroperation triggering the editing control 8 on the target page, thepermission editing list may be popped up, and a permission of eachinteractive user may be edited based on the permission editing list.

It should be noted that in the process of the practical application, ifthe multimedia data stream and the corresponding display text need to beshared to other users, it may be achieved through a sharing control onthe target page. For example, in response to detecting a triggeringoperation triggering the sharing control on the target page, a targetlink corresponding to the current target page is generated, and thetarget link is sent to a user to share with, to cause the user to sharewith to view the multimedia data stream and the display text based onthe target link.

The sharing control may share the multimedia data stream and the displaytext included in the target page to other users.

For example, in response to detecting a triggering operation triggeringthe sharing control on the target page, a link based on the content onthe target page may be generated and sent to other users. When the linkis received by other users, the target page including the multimediadata stream and the display text may be acquired based on the link.

Exemplarily, referring to FIG. 3, in response to detecting a triggeringoperation triggering a sharing control 9 on the target page, a targetlink corresponding to the target page may be generated, and the targetlink is sent to other users to cause the other users to view themultimedia data stream and the display text based on the target link.

On the basis of the technical schemes, it should be noted that in orderto make other users to be clear about each interactive user and thetotal number of users in the multimedia data stream, a head portrait ofeach interactive user and an identity of the total number of users maybe displayed, for example, at a position corresponding to a control 10,and the host of the multimedia data stream may also be displayed, forexample, at a position corresponding to a control 11.

FIG. 8 is a flowchart of an interactive information display methodaccording to an embodiment of the present disclosure. As shown in FIG.8, the method includes the steps below.

In S810, a display text generated based on a multimedia data stream isdisplayed on a first display region of a target page.

In S820, the multimedia data stream is displayed on a second displayregion of the target page.

A position correspondence is provided between the display text and themultimedia data stream.

On the basis of the preceding technical schemes, the method furtherincludes: displaying, on a third display region of the target page, thecontent corresponding to a video frame of the multimedia data stream,and displaying, on a fourth display region of the target page, at leastone of a keyword or a key sentence in an audio-video frame of themultimedia data stream. The audio-video frame includes the video frame.

On the basis of the preceding technical schemes, the method furtherincludes, in a playback process of the multimedia data stream,displaying the display content in a display text corresponding to thecurrent playback progress sequentially and differentially based on theposition correspondence between the display text and the multimedia datastream. The differentially displaying may be color-differentiallydisplaying, highlight displaying and so on.

On the basis of the preceding technical schemes, the method furtherincludes, in response to a triggering operation for a playback progressof the multimedia data stream, performing a jumping display on thedisplay content of the display text based on the positioncorrespondence, or in response to a triggering operation for the displaycontent of the display text, performing a jumping display on theplayback progress of the multimedia data stream based on the positioncorrespondence.

FIG. 9 is a structural diagram of an interactive information processingapparatus according to an embodiment of the present disclosure. As shownin FIG. 9, the apparatus includes a correspondence establishment module610 and a content presentation module.

The correspondence establishment module 610 is configured to establish aposition correspondence between a display text generated based on amultimedia data stream and the multimedia data stream, and the contentpresentation module 620 is configured to present the display text andthe multimedia data stream corresponding to the display text based onthe position correspondence. On the basis of the preceding technicalschemes, the apparatus further includes a user identity acquisitionmodule and a display text determination module. The user identityacquisition module is configured to acquire an audio-video frame of themultimedia data stream, and determine a user identity of a speaking usercorresponding to the audio-video frame. The display text determinationmodule is configured to generate a display text corresponding to themultimedia data stream based on the user identity and the audio-videoframe.

On the basis of the preceding technical schemes, the method furtherincludes an information determination module. The informationdetermination module is configured to determine the user identity of thespeaking user by performing the voiceprint recognition on the audioframe; and/or determine a client identity of a client to which the audioframe belongs, and determine the user identity of the speaking userbased on the client identity.

On the basis of the preceding technical schemes, the apparatus furtherincludes a first text display module.

The first text display module is configured to obtain a literalexpression corresponding to the audio frame by performing thespeech-to-text processing on the audio frame, and generate a firstdisplay text in the display text based on the literal expression and theuser identity.

On the basis of the preceding technical schemes, the apparatus furtherincludes a second text display module. The second text display module isconfigured to obtain characters in the video frame by performing theimage-text recognition on the video frame, and generate a second displaytext in the display text based on the characters and the user identity.

On the basis of the preceding technical schemes, the first display textdetermination module is further configured to determine the literalexpression corresponding to the audio frame, a timestamp currentlycorresponding to the audio frame and a user identity of a speaking userto which the audio frame belongs, and generate the display content inthe display text based on the user identity, the timestamp and theliteral expression.

On the basis of the preceding technical schemes, each piece of displaycontent includes at least one paragraph, and the first text displaymodule is configured to, in response to detecting that an intervalduration between adjacent audio frames is greater than or equal to apreset interval duration threshold and a user identity of a latter audioframe of the adjacent audio frames is not changed, generate the nextparagraph in the display content based on the latter audio frame in aprocess of performing the speech-to-text processing based on the audioframe.

On the basis of the preceding technical schemes, the apparatus furtherincludes a second text display module. The second text display module isconfigured to, in response to determining that the video frame includesat least one URL address, generate a first display content in the seconddisplay text based on the at least one URL address.

On the basis of the preceding technical schemes, the apparatus furtherincludes a second text display module. The second text display module isfurther configured to, in response to determining that the video frameincludes a character, determine a second display content in the seconddisplay text based on the character.

On the basis of the preceding technical schemes, the display textdetermination module is configured to determine a third display text inthe display text based on the audio-video frame to determine a contentcorresponding to a target content from the first display text inresponse to detecting that the target content in the third display textis triggered, and display the content differentially. The third displaytext includes at least one keyword and/or at least one key sentence.

On the basis of the preceding technical schemes, displaying the displaytext and a recording screen video on the target page includes displayingthe third display text, the second display text, the first display textin the display text and the recording screen video in preset displayregions on the target page, respectively.

On the basis of the preceding technical schemes, the apparatus includesa region proportion determination module, which is configured todetermine a region proportion of the first display text, a regionproportion of the second display text and a region proportion of thethird display text on a display page based on the display content in thefirst display text, the display content in the second display text andthe display content in the third display text.

On the basis of the preceding technical schemes, the apparatus includesthe display text determination module, which is configured to obtain anoriginal language type corresponding to audio information, and generatethe display text corresponding to the multimedia data stream based onthe user identity, the audio-video frame and the original language typecorresponding to the audio-video frame.

On the basis of the preceding technical schemes, the display textdetermination module is configured to determine a target language type,and convert the audio-video frame from an original language typecorresponding to the audio-video frame into a literal expressioncorresponding to the target language type; and generate the display textbased on the literal expression and the user identity.

On the basis of the preceding technical schemes, the target languagetype determination module is configured to acquire a historical languagetype used by a current client, and determine the target language typebased on the historical language type.

On the basis of the preceding technical schemes, determining the targetlanguage type based on the historical language type includes at leastone of: determining the target language type from the at least onehistorical language type based on a use frequency of each of at leastone historical language type; using a language type set by the currentclient as the target language type; or determining the target languagetype based on a login address of the current client.

On the basis of the preceding technical schemes, the positioncorrespondence includes a timestamp synchronization association. Themodule is further configured to establish the timestamp synchronizationassociation between the display text and the multimedia data streambased on a timestamp in the display text and a timestamp of themultimedia data stream.

On the basis of the preceding technical schemes, presenting the displaytext and the multimedia data stream corresponding to the display textbased on the position correspondence includes displaying the displaycontent corresponding to the multimedia data stream differentially inthe display text based on the position correspondence.

On the basis of the preceding technical schemes, the multimedia datastream and the display text are displayed on a target page, and thetarget page further includes a controlling control. The apparatusfurther includes a control module, which is configured to adjust thecurrently displayed content of the display text and the recording screenvideo simultaneously based on the controlling control. The adjustmentprecision corresponding to the controlling control is greater than theadjustment precision of an adjustment control for adjusting the progressof the recording screen video in the recording screen video.

On the basis of the preceding technical schemes, the apparatus furtherincludes a control module, which is configured to, in response todetecting a triggering operation triggering the controlling control onthe target page, acquire a playback timestamp corresponding to thecontrolling control, adjust the multimedia data stream to jump to aplayback position corresponding to the playback timestamp based on theplayback timestamp, and display the content of a display text of a videoframe corresponding to the playback timestamp differentially in thedisplay text.

On the basis of the preceding technical schemes, the apparatus furtherincludes a jump module, which is configured to: in response to detectinga triggering operation triggering a display content in the display texton a target page, adjust, based on a timestamp corresponding to thedisplay content, the recording screen video to jump to a video framecorresponding to the timestamp; and/or, in response to detecting atriggering operation for the recording screen video, acquire a playbacktimestamp of the recording screen video corresponding to the triggeringoperation, and jump the display text to a display content correspondingto the playback timestamp in the display text based on the playbacktimestamp.

On the basis of the preceding technical schemes, the apparatus furtherincludes a permission editing module, which is configured to, inresponse to detecting a triggering operation for an editing control on atarget page, display a permission editing list, and determine a userpermission of each interactive user based on the permission editinglist, where the user permission is used for representing an accesspermission of a user to the content presented on the target page.

On the basis of the preceding technical schemes, the apparatus furtherincludes a target page sharing module, which is configured to: inresponse to detecting a triggering operation triggering a sharingcontrol on a target page, generate a target identity corresponding tothe target page, and send the target identity to a user to share with,to cause the user to share with to acquire the target page based on thetarget identity.

On the basis of the preceding technical schemes, the apparatus furtherincludes a search module, which is configured to acquire a searchcontent edited in a search content editing control, and acquire at leastone target content corresponding to the search content from the displaytext; and display the at least one target content differentially in thedisplay text, and mark an audio-video frame corresponding to the atleast one target content in a controlling control corresponding to themultimedia data stream. Each of the at least one target content is thesame as the search content.

On the basis of the preceding technical schemes, the search module isfurther configured to determine a playback timestamp corresponding toeach of the at least one target content, and mark an audio-video framecorresponding to the each of the at least one target content in thecontrolling control corresponding to the multimedia data streamaccording to the playback timestamp.

On the basis of the preceding technical schemes, the apparatus furtherincludes a differential display module, which is configured to, inresponse to detecting a triggering operation triggering each of the atleast one target content, determine a target playback timestamp of theeach of the at least one target content; and display a markercorresponding to the target playback timestamp differentially in thecontrolling control.

According to the technical schemes of the embodiments of the presentdisclosure, a position linkage between the recording screen video andthe display text is achieved by establishing the position correspondencebetween the recording screen video and the display text, therebyimproving the high efficiency and convenience for users to read the textand watch the recording screen video.

The interactive information processing apparatus according to theembodiments of the present disclosure can execute the interactiveinformation processing method according to any embodiment of the presentdisclosure, and has functional modules and beneficial effectscorresponding to the execution methods.

It is to be noted that units and modules included in the precedingapparatus are just divided according to functional logic, and thedivision is not limited to this, as long as the corresponding functionscan be implemented. Additionally, the specific names of the functionalunits are just intended for distinguishing, and are not to limit theprotection scope of embodiments of the present disclosure.

FIG. 10 shows a structure diagram of an electronic device 700 (such asthe terminal device or server in FIG. 10) applicable to implementing anembodiment of the present disclosure. A terminal device in theembodiments of the present disclosure may include, but is not limitedto, mobile terminals such as a mobile phone, a laptop, a digitalbroadcast receiver, a personal digital assistant (PDA), a PAD, aportable media player (PMP), and an in-vehicle terminal (such as anin-vehicle navigation terminal) and stationary terminals such as adigital television (TV) and a desktop computer. The electronic deviceshown in FIG. 10 is merely an example and is not intended to limit thefunction and usage scope of embodiments of the present disclosure.

As shown in FIG. 10, the electronic device 700 may include theprocessing apparatus 701 (such as a central processing unit and agraphics processing unit). The processing apparatus 702 may performvarious types of appropriate operations and processing based on aprogram stored in the read-only memory (ROM) 706 or a program loadedfrom the storage apparatus 703 to the random-access memory (RAM) 603.Various programs and data required for the operation of the electronicdevice 700 are also stored in the RAM 703. The processing apparatus 701,the ROM 702, and the RAM 703 are connected through a bus 704. Aninput/output (I/O) interface 705 is also connected to the bus 704.

Generally, the following apparatus may be connected to the I/O interface705: an input apparatus 706 such as a touch screen, a touch pad, akeyboard, a mouse, a camera, a microphone, an accelerometer, and agyroscope; an output apparatus 707 such as a liquid crystal display(LCD), a speaker, and a vibrator; the storage apparatus 706 such as amagnetic tape and a hard disk, and a communication apparatus 709. Thecommunication apparatus 709 may allow the electronic device 700 toperform wireless or wired communication with other devices to exchangedata. Although FIG. 10 shows the electronic device 700 having variousapparatuses, it is to be understood that all of the apparatuses shownherein need not be implemented or present. Alternatively, more or fewerapparatuses may be implemented or present.

Particularly, according to the embodiment of the present disclosure, theprocess described preceding with reference to the flowchart may beimplemented as a computer software program. For example, the embodimentof the present disclosure includes a computer program product. Thecomputer program product includes a computer program carried in anon-transitory computer-readable medium. The computer program includesprogram codes for performing the method shown in the flowchart. In suchan embodiment, the computer program may be downloaded from a network andinstalled through the communication apparatus 709, or may be installedfrom the storage apparatus 706, or may be installed from the ROM 702.When the computer program is executed by the processing apparatus 701,the preceding functions defined in the methods of the embodiments of thepresent disclosure are performed.

The electronic device provided in this embodiment belongs to the sameinventive concept as the interactive information processing methodprovided in the preceding-mentioned embodiment, and for the technicaldetails not described in detail in this embodiment, reference can bemade to the preceding-mentioned embodiment, and this embodiment has thesame beneficial effects as the preceding embodiments.

This embodiment provides a computer storage medium. A computer programis stored on the computer storage medium. When the program is executedby a processor, the interactive information processing method providedin the preceding-mentioned embodiment is implemented.

It is to be noted that the preceding computer-readable medium in thepresent disclosure may be a computer-readable signal medium or acomputer-readable storage medium or any combination thereof. Thecomputer-readable storage medium may be, but is not limited to, anelectrical, magnetic, optical, electromagnetic, infrared orsemiconductor system, apparatus or device, or any combination thereof.More specific examples of the computer-readable storage medium mayinclude, but are not limited to, an electrical connection with one ormore wires, a portable computer magnetic disk, a hard disk, arandom-access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM), a flash memory, an optical fiber,a portable compact disk read-only memory (CD-ROM), an optical memorydevice, a magnetic memory device, or any appropriate combinationthereof. In the present disclosure, the computer-readable storage mediummay be any tangible medium including or storing a program. The programmay be used by or used in conjunction with an instruction executionsystem, apparatus, or device. In the present disclosure, thecomputer-readable signal medium may include a data signal propagated ona baseband or as a part of a carrier, and computer-readable programcodes are carried in the data signal. The data signal propagated in thismanner may be in multiple forms and includes, and is not limited to, anelectromagnetic signal, an optical signal, or any suitable combinationthereof. The computer-readable signal medium may further be anycomputer-readable medium other than the computer-readable storagemedium. The computer-readable signal medium may send, propagate, ortransmit a program used by or in conjunction with an instructionexecution system, apparatus, or device. The program codes included onthe computer-readable medium may be transmitted via any appropriatemedium which includes, but is not limited to, a wire, an optical cable,a radio frequency (RF), or any appropriate combination thereof.

In some embodiments, clients and servers may communicate using anycurrently known or future developed network protocol, such as thehypertext transfer protocol (HTTP), and may be interconnected with anyform or medium of digital data communication (such as a communicationnetwork). Examples of the communication network include a local areanetwork (LAN), a wide area network (WAN), an internet (such as theInternet) and a peer-to-peer network (such as an ad hoc network), aswell as any currently known or future developed network.

The computer-readable medium may be included in the electronic device ormay exist alone without being assembled into the electronic device.

The computer-readable medium carries one or more programs. When the oneor more programs are executed by the electronic device, the electronicdevice is configured to establish a position correspondence between adisplay text generated based on a multimedia data stream and themultimedia data stream; and present the display text and the multimediadata stream corresponding to the display text based on the positioncorrespondence.

Computer program codes for performing the operations in the presentdisclosure may be written in one or more programming languages or acombination thereof. The preceding one or more programming languagesinclude, but are not limited to, object-oriented programming languagessuch as Java, Smalltalk and C++, as well as conventional proceduralprogramming languages such as C or similar programming languages. Theprogram codes may be executed entirely on a user computer, partly on theuser computer, as a stand-alone software package, partly on the usercomputer and partly on a remote computer, or entirely on the remotecomputer or a server. In the case relating to the remote computer, theremote computer may be connected to the user computer via any type ofnetwork including a local area network (LAN) or a wide area network(WAN), or may be connected to an external computer (for example, via theInternet through an Internet service provider).

The flowcharts and block diagrams in the drawings show the possiblearchitecture, function and operation of the system, method and computerprogram product according to various embodiments of the presentdisclosure. In this regard, each block in a flowchart or block diagrammay represent a module, a program segment, or part of codes thatcontains one or more executable instructions for implementing specifiedlogical functions. It is also to be noted that in some alternativeimplementations, the functions in the blocks may occur in sequencesdifferent from those shown in the drawings. For example, two successiveblocks may actually be executed substantially in parallel or in areverse order, which depends on the functions involved. It is also to benoted that each block in the block diagrams and/or flowcharts and acombination of blocks in the block diagrams and/or flowcharts may beimplemented by a specific-purpose hardware-based system which performsspecified functions or operations or a combination of specific-purposehardware and computer instructions.

The units described in the embodiments of the present disclosure may beimplemented in software or hardware. The names of the modules/units donot constitute a limitation on the units themselves. For example, acorrespondence establishment module may also be described as “atimestamp synchronization association establishment module”.

The functions described preceding herein may be performed, at leastpartially, by one or more hardware logic components. For example, andwithout limitations, example types of hardware logic components that maybe used include: a field-programmable gate array (FPGA), anapplication-specific integrated circuit (ASIC), an application-specificstandard product (ASSP), a system on a chip (SOC), a complexprogrammable logic device (CPLD) and the like.

In the context of the present disclosure, a machine-readable medium maybe a tangible medium that may include or store a program that is used byor in conjunction with an instruction execution system, apparatus ordevice. The machine-readable medium may be a machine-readable signalmedium or a machine-readable storage medium. The machine-readable mediummay include, but is not limited to, an electronic, magnetic, optical,electromagnetic, infrared or semiconductor system, apparatus or device,or any suitable combination thereof. Concrete examples of themachine-readable storage medium may include an electrical connectionbased on one or more wires, a portable computer disk, a hard disk, arandom-access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM) or a flash memory, an opticalfiber, a portable compact disc read-only memory (CD-ROM), an opticalstorage device, a magnetic storage device, or any appropriatecombination thereof.

According to one or more embodiments of the present disclosure, [example1] provides an interactive information processing method. The methodincludes: establishing a position correspondence between a display textgenerated based on a multimedia data stream and the multimedia datastream; and presenting the display text and the multimedia data streamcorresponding to the display text based on the position correspondence.

According to one or more embodiments of the present disclosure, [example2] provides an interactive information processing method. The methodfurther includes the following.

For example, an audio-video frame of the multimedia data stream isacquired, and g a user identity of a speaking user corresponding to theaudio-video frame is determined; and a display text corresponding to themultimedia data stream is generated based on the user identity and theaudio-video frame.

According to one or more embodiments of the present disclosure, [example3] provides an interactive information processing method. The methodfurther includes the following.

For example, acquiring the audio-video frame of the multimedia datastream, and determining the user identity of the speaking usercorresponding to the audio-video frame include at least one ofdetermining the user identity of the speaking user by performing avoiceprint recognition on an audio frame; or determining a clientidentity of a client to which the audio frame belongs, and determiningthe user identity of the speaking user based on the client identity.

According to one or more embodiments of the present disclosure, [example4] provides an interactive information processing method. The methodfurther includes the following.

For example, generating the display text corresponding to the multimediadata stream based on the user identity and the audio-video frameincludes: obtaining a literal expression corresponding to the audioframe by performing the speech-to-text processing on the audio frame,and generating a first display text in the display text based on theliteral expression and the user identity.

According to one or more embodiments of the present disclosure, [example5] provides an interactive information processing method. The methodfurther includes the following.

For example, generating the display text corresponding to the multimediadata stream based on the user identity and the audio-video frameincludes: obtaining characters in the video frame by performing animage-text recognition on the video frame, and generating a seconddisplay text in the display text based on the characters and the useridentity.

According to one or more embodiments of the present disclosure, [example6] provides an interactive information processing method. The methodfurther includes the following.

For example, obtaining the literal expression corresponding to the audioframe by performing the speech-to-text processing on the audio frame,and generating the first display text in the display text based on theliteral expression and the user identity include: determining theliteral expression corresponding to the audio frame, a timestampcurrently corresponding to the audio frame and a user identity of aspeaking user to which the audio frame belongs; and generating a pieceof display content in the display text based on the user identity, thetimestamp and the literal expression.

According to one or more embodiments of the present disclosure, [example7] provides an interactive information processing method. The methodfurther includes the following.

For example, each piece of display content includes at least oneparagraph; and obtaining the literal expression corresponding to theaudio frame by performing the speech-to-text processing on the audioframe, and generating the first display text in the display text basedon the literal expression and the user identity include: in a process ofperforming the speech-to-text processing based on the audio frame, inresponse to detecting that an interval duration between adjacent audioframes is greater than or equal to a preset interval duration thresholdand a user identity of a latter audio frame of the adjacent audio framesis not changed, generating a next paragraph in the display content basedon the latter audio frame.

According to one or more embodiments of the present disclosure, [example8] provides an interactive information processing method. The methodfurther includes the following.

For example, obtaining the second display text in the display text byperforming the image-text recognition on the video frame includes: inresponse to determining that the video frame comprises at least oneuniform resource locator (URL) address, generating a first displaycontent in the second display text based on the at least one URLaddress.

According to one or more embodiments of the present disclosure, [example9] provides an interactive information processing method. The methodfurther includes the following.

For example, obtaining the second display text in the display text byperforming the image-text recognition on the video frame includes, inresponse to determining that the video frame comprises a character,determining a second display content in the second display text based onthe character.

According to one or more embodiments of the present disclosure, [example10] provides an interactive information processing method. The methodfurther includes the following.

For example, generating the display text corresponding to the multimediadata stream based on the user identity and the audio-video frameincludes: determining a third display text in the display text based onthe audio-video frame to determine a content corresponding to a targetcontent from the first display text in response to detecting that thetarget content in the third display text is triggered, and display thecontent differentially; where the third display text includes at leastone keyword and/or at least one key sentence.

According to one or more embodiments of the present disclosure, [example11] provides an interactive information processing method. The methodfurther includes the following.

For example, displaying the display text and the multimedia data streamon the target page includes: displaying the third display text, thesecond display text, the first display text in the display text and arecording screen video in preset display regions on the target page,respectively.

According to one or more embodiments of the present disclosure, [example12] provides an interactive information processing method. The methodfurther includes the following.

For example, a region proportion of the first display text, a regionproportion of the second display text and a region proportion of thethird display text on a display page are determined based on the displaycontent in the first display text, the display content in the seconddisplay text and the display content in the third display text.

According to one or more embodiments of the present disclosure, [example13] provides an interactive information processing method. The methodfurther includes the following.

For example, generating the display text corresponding to the multimediadata stream based on the user identity and the audio-video frameincludes: obtaining an original language type corresponding to audioinformation in the audio-video frame; and generating the display textcorresponding to the multimedia data stream based on the user identity,the audio-video frame and the original language type corresponding tothe audio-video frame.

According to one or more embodiments of the present disclosure, [example14] provides an interactive information processing method. The methodfurther includes the following.

For example, generating the display text corresponding to the multimediadata stream based on the user identity and the audio-video frameincludes: determining a target language type, and converting theaudio-video frame from an original language type corresponding to theaudio-video frame to a literal expression corresponding to the targetlanguage type; and generating the display text based on the literalexpression and the user identity.

According to one or more embodiments of the present disclosure, [example15] provides an interactive information processing method. The methodfurther includes the following.

For example, determining the target language type includes acquiring ahistorical language type used by a current client and determining thetarget language type based on the historical language type.

According to one or more embodiments of the present disclosure, [example16] provides an interactive information processing method. The methodfurther includes the following.

For example, determining the target language type based on thehistorical language type includes at least one of determining the targetlanguage type from the at least one historical language type based on ause frequency of each of at least one historical language type; using alanguage type set by the current client as the target language type; ordetermining the target language type based on a login address of thecurrent client.

According to one or more embodiments of the present disclosure, [example17] provides an interactive information processing method. The methodfurther includes the following.

For example, the position correspondence comprises a timestampsynchronization association, and establishing the positioncorrespondence between the display text generated based on themultimedia data stream and the multimedia data stream includesestablishing the timestamp synchronization association between thedisplay text and the multimedia data stream based on a timestamp in thedisplay text and a timestamp of the multimedia data stream.

According to one or more embodiments of the present disclosure, [example18] provides an interactive information processing method. The methodfurther includes the following.

For example, presenting the display text and the multimedia data streamcorresponding to the display text based on the position correspondenceincludes displaying the display content corresponding to the multimediadata stream differentially in the display text based on the positioncorrespondence.

According to one or more embodiments of the present disclosure, [example19] provides an interactive information processing method. The methodfurther includes the following.

For example, the multimedia data stream and the display text aredisplayed on a target page, the target page further includes acontrolling control; and the method further includes adjusting thecurrently displayed content of the display text and the recording screenvideo simultaneously based on the controlling control.

An adjustment precision corresponding to the controlling control isgreater than an adjustment precision of an adjustment control foradjusting a progress of the recording screen video in the recordingscreen video.

According to one or more embodiments of the present disclosure, [example20] provides an interactive information processing method. The methodfurther includes the following.

For example, in response to detecting a triggering operation triggeringthe controlling control on the target page, a playback timestampcorresponding to the controlling control is acquired, the multimediadata stream is adjusted to jump to a playback position corresponding tothe playback timestamp based on the playback timestamp, and the contentof a display text of a video frame corresponding to the playbacktimestamp is differentially displayed in the display text.

According to one or more embodiments of the present disclosure, [example21] provides an interactive information processing method. The methodfurther includes the following.

For example, in response to detecting a triggering operation triggeringa display content in the display text on a target page, based on atimestamp corresponding to the display content, the recording screenvideo is adjusted to jump to a video frame corresponding to thetimestamp; and/or, in response to detecting a triggering operation forthe recording screen video, a playback timestamp of the recording screenvideo corresponding to the triggering operation is acquired, and thedisplay text is jumped to a display content corresponding to theplayback timestamp in the display text based on the playback timestamp.

According to one or more embodiments of the present disclosure, [example22] provides an interactive information processing method. The methodfurther includes the following.

For example, in response to detecting a triggering operation for anediting control on a target page, a permission editing list isdisplayed, and a user permission of each interactive user is determinedbased on the permission editing list, where the user permission is usedfor representing an access permission of a user to the content presentedon the target page.

According to one or more embodiments of the present disclosure, [example23] provides an interactive information processing method. The methodfurther includes the following.

For example, in response to detecting a triggering operation triggeringa sharing control on a target page, generating a target identitycorresponding to the target page, and sending the target identity to auser to share with, to cause the user to share with to acquire thetarget page based on the target identity.

According to one or more embodiments of the present disclosure, [example24] provides an interactive information processing method. The methodfurther includes the following.

For example, acquiring a search content edited in a search contentediting control, and acquiring at least one target content correspondingto the search content from the display text; where each of the at leastone target content is the same as the search content; and displaying theat least one target content differentially in the display text, andmarking an audio-video frame corresponding to the at least one targetcontent in a controlling control corresponding to the multimedia datastream.

According to one or more embodiments of the present disclosure, [example25] provides an interactive information processing method. The methodfurther includes the following.

For example, marking the audio-video frame corresponding to the at leastone target content in the controlling control corresponding to themultimedia data stream includes: determining a playback timestampcorresponding to each of the at least one target content, and marking anaudio-video frame corresponding to the each of the at least one targetcontent in the controlling control corresponding to the multimedia datastream according to the playback timestamp.

According to one or more embodiments of the present disclosure, [example26] provides an interactive information processing method. The methodfurther includes the following.

For example, in response to detecting a triggering operation triggeringeach of the at least one target content, a target playback timestamp ofthe each of the at least one target content is determined; and a markercorresponding to the target playback timestamp is differentiallydisplayed in the controlling control.

According to one or more embodiments of the present disclosure, [example27] provides an interactive information processing apparatus. Theapparatus includes a correspondence establishment module and a contentpresentation module.

The correspondence establishment module, which is configured toestablish a position correspondence between a display text generatedbased on a multimedia data stream and the multimedia data stream.

The content presentation module is configured to present the displaytext and the multimedia data stream corresponding to the display textbased on the position correspondence.

In addition, although the operations are described in a particularorder, this should not be construed as requiring that such operationsshould be performed in the particular order shown or in a sequentialorder. In certain circumstances, multitasking and parallel processingmay be advantageous. Similarly, although implementation details areincluded in the preceding discussion, these should not be construed aslimiting the scope of the present disclosure. Some features described inthe context of separate embodiments may also be implemented incombination in a single embodiment. Conversely, various featuresdescribed in the context of a single embodiment may also be implementedin multiple embodiments, individually, or in any suitablesub-combination.

What is claimed is:
 1. An interactive information processing method,comprising: establishing a position correspondence between a displaytext generated based on a multimedia data stream and the multimedia datastream; and presenting the display text and the multimedia data streamcorresponding to the display text based on the position correspondence.2. The method of claim 1, further comprising: acquiring an audio-videoframe of the multimedia data stream, and determining a user identity ofa speaking user corresponding to the audio-video frame; and generatingthe display text corresponding to the multimedia data stream based onthe user identity and the audio-video frame.
 3. The method of claim 2,wherein the audio-video frame comprises an audio frame; and acquiringthe audio-video frame of the multimedia data stream, and determining theuser identity of the speaking user corresponding to the audio-videoframe comprise at least one of: determining the user identity of thespeaking user by performing a voiceprint recognition on the audio frame;or determining a client identity of a client to which the audio framebelongs, and determining the user identity of the speaking user based onthe client identity.
 4. The method of claim 2, wherein the audio-videoframe comprises an audio frame; and generating the display textcorresponding to the multimedia data stream based on the user identityand the audio-video frame comprises: obtaining a literal expressioncorresponding to the audio frame by performing a speech-to-textprocessing on the audio frame, and generating a first display text inthe display text based on the literal expression and the user identity.5. The method of claim 2, wherein the audio-video frame comprises avideo frame; and generating the display text corresponding to themultimedia data stream based on the user identity and the audio-videoframe comprises: obtaining characters in the video frame by performingan image-text recognition on the video frame, and generating a seconddisplay text in the display text based on the characters and the useridentity.
 6. The method of claim 4, wherein obtaining the literalexpression corresponding to the audio frame by performing thespeech-to-text processing on the audio frame, and generating the firstdisplay text in the display text based on the literal expression and theuser identity comprise: determining the literal expression correspondingto the audio frame, a timestamp currently corresponding to the audioframe and a user identity of a speaking user to which the audio framebelongs; and generating a display content in the display text based onthe user identity, the timestamp and the literal expression; wherein thedisplay content comprises at least one paragraph; and obtaining theliteral expression corresponding to the audio frame by performing thespeech-to-text processing on the audio frame, and generating the firstdisplay text in the display text based on the literal expression and theuser identity comprise: in a process of performing the speech-to-textprocessing based on the audio frame, in response to detecting that aninterval duration between adjacent audio frames is greater than or equalto a preset interval duration threshold and a user identity of a latteraudio frame of the adjacent audio frames is not changed, generating anext paragraph in the display content based on the latter audio frame.7. The method of claim 5, wherein obtaining the second display text inthe display text by performing the image-text recognition on the videoframe comprises at least one of: in response to determining that thevideo frame comprises at least one uniform resource locator (URL)address, generating a first display content in the second display textbased on the at least one URL address; or in response to determiningthat the video frame comprises a character, determining a second displaycontent in the second display text based on the character.
 8. The methodof claim 4, wherein generating the display text corresponding to themultimedia data stream based on the user identity and the audio-videoframe comprises: determining, based on the audio-video frame, a thirddisplay text in the display text to determine a content corresponding toa target content from the first display text in response to detectingthat the target content in the third display text is triggered, anddisplay the content differentially; wherein the third display textcomprises at least one of at least one keyword or at least one keysentence.
 9. The method of claim 8, wherein the display text comprises asecond display text; and the method further comprises: displaying thedisplay text and the multimedia data stream on a target page; whereindisplaying the display text and the multimedia data stream on the targetpage comprises: displaying the third display text, the second displaytext, and the first display text in the display text and a recordingscreen video in preset display regions on the target page, respectively.10. The method of claim 9, further comprising: determining a regionproportion of the first display text, a region proportion of the seconddisplay text and a region proportion of the third display text on thetarget page based on a display content in the first display text, adisplay content in the second display text and a display content in thethird display text.
 11. The method of claim 2, wherein generating thedisplay text corresponding to the multimedia data stream based on theuser identity and the audio-video frame comprises: obtaining an originallanguage type corresponding to audio information in the audio-videoframe; and generating the display text corresponding to the multimediadata stream based on the user identity, the audio-video frame and theoriginal language type corresponding to the audio-video frame.
 12. Themethod of claim 2, wherein generating the display text corresponding tothe multimedia data stream based on the user identity and theaudio-video frame comprises: determining a target language type, andconverting the audio-video frame from an original language typecorresponding to the audio-video frame to a literal expressioncorresponding to the target language type; and generating the displaytext based on the literal expression and the user identity.
 13. Themethod of claim 12, wherein determining the target language typecomprises: acquiring a historical language type used by a currentclient, and determining the target language type based on the historicallanguage type; wherein the historical language type comprises at leastone language type; and determining the target language type based on thehistorical language type comprises at least one of: determining thetarget language type from the at least one historical language typebased on a use frequency of each of the at least one historical languagetype; using a language type set by the current client as the targetlanguage type; or determining the target language type based on a loginaddress of the current client.
 14. The method of claim 1, wherein theposition correspondence comprises a timestamp synchronizationassociation, and establishing the position correspondence between thedisplay text generated based on the multimedia data stream and themultimedia data stream comprises: establishing the timestampsynchronization association between the display text and the multimediadata stream based on a timestamp of the display text and a timestamp ofthe multimedia data stream.
 15. The method of claim 1, whereinpresenting the display text and the multimedia data stream correspondingto the display text based on the position correspondence comprises:displaying a display content corresponding to the multimedia data streamdifferentially in the display text based on the position correspondence.16. The method of claim 1, wherein the multimedia data stream and thedisplay text are displayed on a target page, and the target page furthercomprises a controlling control; and the method further comprises:adjusting a currently displayed content of the display text and themultimedia data stream simultaneously based on the controlling control;wherein an adjustment precision corresponding to the controlling controlis greater than an adjustment precision of an adjustment control foradjusting a progress of a recording screen video in the multimedia datastream.
 17. The method of claim 16, further comprising: in response todetecting a triggering operation triggering the controlling control onthe target page, acquiring a playback timestamp corresponding to thecontrolling control, adjusting the multimedia data stream to jump to aplayback position corresponding to the playback timestamp based on theplayback timestamp, and displaying a content of a display text of avideo frame corresponding to the playback timestamp differentially inthe display text.
 18. The method of claim 1, further comprising at leastone of: in response to detecting a triggering operation triggering adisplay content in the display text on a target page, adjusting, basedon a timestamp corresponding to the display content, the multimedia datastream to jump to a video frame corresponding to the timestamp; inresponse to detecting a triggering operation for the multimedia datastream, acquiring a playback timestamp of the multimedia data streamcorresponding to the triggering operation, and jumping the display textto a display content corresponding to the playback timestamp in thedisplay text based on the playback timestamp; in response to detecting atriggering operation for an editing control on a target page, displayinga permission editing list, and determining a user permission of eachinteractive user based on the permission editing list, wherein the userpermission is used for representing an access permission of a user to acontent presented on the target page, and the permission editing listcomprises at least one user permission of an interactive user; or inresponse to detecting a triggering operation triggering a sharingcontrol on a target page, generating a target identity corresponding tothe target page, and sending the target identity to a user to sharewith, to cause the user to share with to acquire the target page basedon the target identity.
 19. The method of claim 2, further comprising:acquiring a search content edited in a search content editing control,and acquiring at least one target content corresponding to the searchcontent from the display text; wherein each of the at least one targetcontent is the same as the search content; and displaying the at leastone target content differentially in the display text, and marking anaudio-video frame corresponding to the at least one target content in acontrolling control corresponding to the multimedia data stream.
 20. Themethod of claim 19, wherein marking the audio-video frame correspondingto the at least one target content in the controlling controlcorresponding to the multimedia data stream comprises: determining aplayback timestamp corresponding to each of the at least one targetcontent, and marking an audio-video frame corresponding to the each ofthe at least one target content in the controlling control correspondingto the multimedia data stream according to the playback timestamp. 21.The method of claim 19, further comprising: in response to detecting atriggering operation triggering each of the at least one target content,determining a target playback timestamp of the each of the at least onetarget content; and displaying a marker corresponding to the targetplayback timestamp differentially in the controlling control.
 22. Aninteractive information display method, comprising: displaying, on afirst display region of a target page, a display text generated based ona multimedia data stream; and displaying the multimedia data stream on asecond display region of the target page; wherein a positioncorrespondence is provided between the display text and the multimediadata stream.
 23. The method of claim 22, further comprising: displaying,on a third display region of the target page, a content corresponding toa video frame of the multimedia data stream; and displaying, on a fourthdisplay region of the target page, at least one of a keyword or a keysentence in an audio-video frame of the multimedia data stream; whereinthe audio-video frame comprises the video frame.
 24. The method of claim22, further comprising: in a playback process of the multimedia datastream, displaying a display content corresponding to a current playbackprogress in the display text sequentially and differentially based onthe position correspondence between the display text and the multimediadata stream.
 25. The method of claim 22, further comprising: in responseto a triggering operation for a playback progress of the multimedia datastream, performing a jumping display on a display content of the displaytext based on the position correspondence; or in response to atriggering operation for a display content of the display text,performing a jumping display on a playback progress of the multimediadata stream based on the position correspondence.
 26. An electronicdevice, comprising: at least one processor; and a storage apparatus,which is configured to store at least one program; wherein the at leastone program, when executed by the at least one processor, causes the atleast one processor to implement the following: establishing a positioncorrespondence between a display text generated based on a multimediadata stream and the multimedia data stream; and presenting the displaytext and the multimedia data stream corresponding to the display textbased on the position correspondence.
 27. A non-transitory storagemedium, comprising computer-executable instructions which, when executedby a computer processor, are configured to implement the method of claim1.